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ABSTRACT 

Socially responsible investing (SRI) represents an investment process that reflects environ- 
mental and social preferences. The financial industry is in a unique position to move cor- 
porations towards corporate sustainability. However, there is often little transparency 
regarding the metrics used to evaluate corporate social and environmental performance 
and the trade-offs involved in the evaluation. In this paper we discuss the various trade-offs 
of sustainability screening methodologies. We show that the rating of companies varies 
significantly according to whether the screening is based on toxic releases and regulatory 
compliance or on the quality of environmental policy and disclosure. We base our analysis 
on the evaluation of the performance of 15 firms in the chemical sector. The analysis indi- 
cates that firms that have the most advanced reporting and environmental management 
practices tend also to have higher levels of toxic releases and lower environmental compli- 
ance. We provide methodological recommendations to help stakeholders evaluate corpo- 
rate environmental performance. Copyright © 2010 John Wiley & Sons, Ltd and ERP 
Environment. 
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Introduction 


OCIALLY RESPONSIBLE INVESTING (SRI) REPRESENTS AN INVESTMENT PROCESS THAT USES SCREENS BASED ON 

environmental and social preferences to select or avoid investing in companies (Renneboog et al., 2008). 

Many investment research companies have screening methodologies to help socially responsible investors 

select companies. Examples of these companies include KLD Research & Analytics, Inc., based in Boston 
in the US, and the Sustainable Asset Management (SAM) Group, based in Zurich in Switzerland. Each of these 
companies has developed its own research and screening methodology for including companies in its social 
responsibility indexes. More recently, some environmental non-governmental organizations (NGOs) have also 
started to screen companies based on their environmental performance and to propose their own SRI funds. For 
example, the Sierra Club Fund invests in socially and environmentally progressive US companies as determined 
by 20 environmental and social screens.’ 


* Correspondence to: Magali Delmas, Institute of the Environment and Anderson School of Management, La Kretz Hall, University of California, 
Los Angeles, CA 90095-1496, USA. E-mail: delmas@ucla.edu 
*http://www.sierraclubfunds.com/ [1 March 2009]. 
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The financial sector is in a unique position to move corporations towards corporate sustainability (Levine and 
Chatterji, 2006; O’Rourke, 2003). This is because socially responsible firms included in these screens might be 
in a better position to attract capital. It is estimated that almost 11 percent of the assets under professional manage- 
ment in the United States are invested with social responsibility in mind (Social Investment Forum, 2007). This 
amounts to $2.71 trillion in total assets under management using one or more of the three core socially respon- 
sible investing strategies: screening, shareholder advocacy and community investing. In the past two years, social 
investing has enjoyed healthy growth, having increased from $2.29 trillion in 2005 (Social Investment Forum, 
2007). As SRI indexes become more prevalent, firms might compete to be part of such indexes by improving their 
environmental and social performance. 

At the core of socially responsible investing is a fundamental question: can good environmental performance 
be associated with good financial performance? While there is an important literature analyzing the link between 
environmental and financial performance, there is still uncertainty about the significance of the relationship 
(Dowell et al., 2000; King and Lenox, 2001; Konar and Cohen, 2001; Margolis and Walsh, 2003; Orlitzky et al., 
2003; Russo and Fouts, 1997; Waddock and Graves, 1997). Some have argued that these mixed results might be 
partly due to the difficulty of measuring environmental performance and to important differences among screen- 
ing methodologies (Griffin and Mahon, 1997). 

Although a growing number of investors are using SRI screening, the methodologies used to evaluate and screen 
corporations are not yet standardized and are often kept confidential by the rating organizations. While financial 
performance indicators are well defined and very structured (for instance return on assets and return on invest- 
ment), environmental performance indicators are quite heterogeneous. This creates a situation in which the results 
of the screening differ widely based on the methodology used (Levine and Chatterji, 2006; Chatterji and Levine, 
2008; O’Rourke, 2003). Because of the use of different metrics, investors might have little confidence in basing 
investment decisions on SRI screens. Metrics that are not comparable could actually lead to outcomes that harm 
corporate social performance (Levine and Chatterji, 2006). Furthermore, corporate managers might be confused 
on how to prioritize their investments in environmental improvements in order to improve the reputation of their 
firm with investors. 

Most importantly, there might be some trade-offs between the different metrics chosen. For example, some 
investors might decide to reward firms that are investing today in new environmental management practices but 
are still major polluters. Others might prefer to invest in companies that currently have a lower impact on the 
environment. A recognition of these trade-offs could help increase stakeholders’ confidence in SRI. 

In this paper, we focus on the environmental dimension of social responsibility and evaluate various criteria to 
compare firms’ environmental performance and management practices. We present a case study based on the 
analysis of the environmental performance of 15 publicly traded companies in the chemical sector. This industry 
was chosen because of the salience of environmental concerns in the chemical industry (Christmann, 2000; 
Delmas and Montiel, 2008; Hoffman, 1999). Our results show how the rating of firms’ environmental perfor- 
mance varies according to the indicators used; we also demonstrate the trade-offs associated with different 
approaches. 

This paper is organized as follows. First, we describe the main metrics used to measure corporate environmen- 
tal performance and the trade-offs involved in sustainability rating. Second, we evaluate the performance of 15 
chemical firms, based on their toxic releases, regulatory compliance and disclosure of environmental performance, 
as a case study to illustrate our argumentation. Third, we provide recommendations to improve screening meth- 
odologies and overcome such trade-offs. 


Measuring Environmental Performance 


Corporate environmental performance indicators are usually divided into three main categories: (1) environmental 
impact (toxicity, emissions, energy use etc.); (2) regulatory compliance (non-compliance status, violation fees, 


*z1 percent of the $25.1 trillion in total assets under management tracked in Nelson Information’s Directory of Investment Managers (Social 
Investment Forum, 2007). 
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number of audits etc.) and (3) organizational processes (environmental accounting, audits, reporting, environmen- 
tal management system etc.) (Ilinitch et al., 1998; Lober, 1996; Wood, 1991). Screening and investment firms use 
different mixes of the above categories. For example, SAM focuses on eco-efficiency and environmental reporting 
along with industry-specific criteria.» KLD Research & Analytics, Inc. analyzes and selects firms based on strengths 
and concerns in the following categories: products and services (beneficial products and services, ozone-depleting 
chemicals, agricultural chemicals); operations and management (pollution prevention, recycling, management 
systems, hazardous waste, regulatory problems, substantial emissions) and climate change (clean energy, revenues 
from coal oil and derivative products) (Waddock and Graves, 1997). 

Socially responsible investors might have different objectives. For some investors, evaluating corporate environ- 
mental performance is another way of evaluating financial performance. For example, Innovest, now part of the 
Riskmetrics group, specializes in analyzing ‘companies’ performance on environmental, social, and strategic gov- 
ernance issues, with a particular focus on their impact on competitiveness, profitability, and share price perfor- 
mance. By assessing differentials typically not identified by traditional securities analysis, Innovest’s ratings 
uncover hidden risks and value potential for investors’.* In this case, corporate environmental performance is a 
proxy for good management. For other investors, the objective is to screen risky investments by filtering out com- 
panies or entire sectors that are exposed to environmental risks. For example, the Sierra Club Mutual Fund screens 
out fossil fuel generation because of its impact on climate change.’ Because of these different objectives, the same 
firm might be rated very differently by different investors. 

As we argue in this article, there are trade-offs associated with the choices of metrics and methodology used in 
SRI. We describe below the main trade-offs involved in measuring and evaluating corporate environmental per- 
formance. 


Trade-Offs Between Positive and Negative Screenings 


Sustainable investors can use negative or positive screening methodologies. Negative screening, also called ‘exclu- 
sionary screening’, relates to the exclusion of companies that do not perform well on some indicators or do belong 
to sectors that might be perceived as having a relatively high impact on the environment. Positive screening seeks 
to identify companies that are the best performers on some indicators. For example SAM, through the Dow Jones 
Sustainability Index, screens companies that are the best in class in their sector on several criteria related to cor- 
porate social responsibility. Negative screenings have been initially favored because it is often easier to agree on 
what constitutes a problem than to agree on what constitutes excellence. Indeed, publicly traded corporations are 
often complex and diversified entities and there is always the possibility that a company that excels on many indi- 
cators also performs poorly on a few indicators that might not be apparent at first. In other words, there is always 
room for criticism. However, negative screening does not identify best-in-class companies that might also perform 
well financially. The trade-off is therefore between focusing on penalizing corporations based on poor performance 
and rewarding corporations based on good performance. 


Trade-Offs Between Environmental and Corporate Performance Criteria 


How to compare progress on one environmental performance criterion with poor records on another criterion is 
an important challenge. For example, should investors value progress on greenhouse gas emissions to the detri- 
ment of toxic releases? The management of some environmental problems might have more direct impact on the 
firm’s bottom line than others. For example, investors might rate more positively firms that manage their green- 
house gas (GHG) emissions well, because of the potential for regulations in that area that will directly impact 
firms’ operations. Furthermore, energy efficiency measures might not only reduce GHGs and the exposure of the 
firm to GHG regulations but also relate directly to the firm’s efficiency by reducing costs (Klassen and McLaugh- 
lin, 1996; Porter and van der Linde, 1995; Reinhardt, 1999). However, in some cases the main impact of such a 


shttp://www.sam-group.com/htmle/research/methodology/steps/analysis.cfm [26 October 2009]. 
+http://www.innovestgroup.com/ [1 September 2008]. 
Shttp://www.businesswire.com/portal/site/google/?ndmViewId=news_view&newsId=20060418005927&newsLang=en [1 March 2009]. 
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firm might be on biodiversity, water consumption or waste generated. The trade-off in this case is to favor envi- 
ronmental issues that might have a more direct and immediate impact on firms’ operations and performance over 
those that might be less directly related to a firm’s operations but could potentially have a bigger environmental 
impact. 


Trade-Offs Between Past, Current and Future Performance 


While most sustainability ratings strive to assess corporate environmental performance, some might choose to 
focus on past or current measured performance while others put the emphasis on the potential to improve future 
performance based on current management practices. For example, SAM claims that its rating methodology is 
performance oriented: ‘While recognizing that approaches to managing certain sustainability issues and challenges 
vary depending on the maturity of the issue, SAM Research constantly aims to evaluate and identify performance 
rather than the mere existence of policies and systems designed to address particular sustainability issues — real 
commitment is demonstrated by performance’.° However, information about actual environmental performance 
is often difficult to obtain, and often indicators of organizational practices or policies are used as proxies for output 
measures. Indicators of organizational practice measures have also the advantage of representing the potential for 
improvement. They indicate the efforts of a company attempting to mitigate its environmental impacts. They might 
therefore be used by some socially responsible investors as a way to predict future corporate environmental per- 
formance. The trade-off here is to focus on management and reporting practices as a proxy for future performance 
at the expense of looking at current performance. 


Trade-Offs Between What Can Be Measured and What Should Be Measured 


Measuring environmental performance faces an important challenge: there is scant data available to compare 
firms. Because the Toxic Release Inventory (TRI) published by the US Environmental Protection Agency (EPA) is 
one of few publicly available sources of comparable data on environmental performance in the US, it has been 
used as the main impact measurement indicator in academic studies (Chatterji et al., 2009; Ilinitch et al., 1998; 
Gerde and Logsdon, 2001; King and Lenox, 2001; Terlaak and King, 2006). Because of this lack of publicly avail- 
able data, socially responsible investors have been complementing TRI information with an analysis of corpora- 
tions’ environmental reports or have been asking corporations directly about their environmental management 
practices and performance through survey questionnaires. However, information provided in environmental 
reports is rarely comparable across firms. Furthermore, firms might be unwilling to respond to survey question- 
naires, because of survey fatigue or because they are reluctant to reveal information regarding their environmen- 
tal emissions (Levine and Chatterji, 2006). There is always the risk that the choice of indicators might be dictated 
by the availability of the data and/or that differences in ratings might be attributable to measurement errors (Chat- 
terji and Levine, 2008). The trade-off is between the ability to compare a higher number of firms with publicly 
available databases that might not always be the most relevant, and using more detailed and important data that 
is possible to obtain for only a smaller subset of these firms. Furthermore, there is an apparent trade-off in the 
resources one might invest in order to collect additional data, and the contribution of such data to the overall 
ranking. 

In conclusion, environmentally conscious investors might have multiple motives and face multiple trade-offs 
when choosing environmental metrics to evaluate corporate environmental performance. We describe below the 
main trade-offs involved in rating corporate environmental performance using the case of 15 chemical companies. 


Case Study 


In order to open the ‘black box’ of sustainability ratings, we present a case study based on the detailed analysis of 
the environmental performance of 15 firms in the chemical sector. Firms were chosen based on the following 


°http://www.sam-group.com/htmle/research/philosophy.cfm [17 September 2008]. 
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Firm Headquarter US sales* Total sales SIC code SIC code description 


location (millions) (millions) 3 digits 
Avon Products, Inc. New York, NY 2141 8 150 284 Soap cleaner and 
Clorox Company Oakland, CA 3 800 4 388 284 toilet goods sector 
Colgate-Palmolive New York, NY 2 507 11 397 284 
Company 
Dial Germany (Henkel) N/A N/A 
Dow Chemical Company Midland, MI 17 597 46 307 282 Plastics materials, 
DuPont Company Wilmington, DE 11 129 27 516 282 synthetic resins, 
Eastman Chemical Kingsport, TN 4 098 7 959 282 and synthetic 
Company rubber sector 
Ecolab Inc. Saint Paul, MN 2 327 4535 284 Soap cleaner and 
toilet goods sector 
International Flavors & New York, NY 572 1993 286 Industrial organic 
Fragrances Inc. chemicals 
Johnson & Johnson New Brunswick, NJ 28 400 50 434 283 Pharmaceutical 
Lilly (Eli) and Company Indianapolis, IN 7 800 14 645 283 preparations 
Merck & Co., Inc. Whitehouse Station, NJ 12 767 22 012 283 
Pfizer, Inc. New York, NY 26 664 51 298 283 
Procter & Gamble Cincinnati, OH 27 236 56 741 284 Soap cleaner and 
Company toilet goods sector 
Rohm and Haas Philadelphia, PA 4 216 7 994 282 Plastics materials, 
Company synthetic resins, 


and synthetic 
rubber sector 


Table 1. Description of firms included in the analysis (2005) 
*Sales in the US or in North America. 


criteria: (1) they belonged to the Standard Industrial Classification code (SIC) 28 for chemical and allied products; 
(2) they had at least one facility reporting to the TRI in 2002; (3) they were publicly traded (and therefore have 
financial information publicly available) and (4) they were included in the KLD database (so we could compare our 
results with others). The cross-sectional analysis is conducted for the year 2005 and the trend analysis is from 
2000 to 2005. We acknowledge that, although all these firms are from the same industrial sector, based on the 
two-digit SIC code, there are still substantial differences between the companies. As can be seen in Table 1, the 
firms analyzed in our study belong to the following main subsectors: cosmetics, pharmaceuticals, cleaning, organic 
chemicals and plastics. However, because our purpose is to demonstrate the trade-offs associated with sustain- 
ability rating, we needed to include companies with different attributes. 


Description of Indicators 


We focus on indicators that provide insights on the three main dimensions of environmental performance men- 
tioned above: toxic releases, compliance with environmental regulations and beyond-compliance management 
practices. We use corporate sales in order to control for variation in size or output.’ 


The Toxic Release Inventory 
The US EPA TRI publishes annual data on releases and transfers of toxic chemicals from US industrial facilities. 
This is a complex database that provides data on approximately 650 toxic chemicals (US EPA, Office of Pollution 


7We use US or North American sales for 2005 since the environmental performance data is US based. Due to data availability, we use total 
sales for the period 2000-2005. The correlation between US and total sales in 2005 is high: 0.975(**). 
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Prevention and Toxics). In this analysis, for the sake of simplicity, we compare firms based on their total pounds 
of toxic releases, which has been the indicator most used by scholars and screening organizations (Chatterji et al., 
2009; Gerde and Logsdon, 2001; Ilinitch et al., 1998; King et al., 2005). 


Risk Screening Environmental Indicators (RSEls) 

One of the limitations of using total pounds from the TRI is that this measure does not provide information about 
the toxicity of the chemical and its potential impact on the population (Toffel and Marshall, 2004). Firms could 
emit small amounts of chemicals that are very toxic and be ranked better than firms with higher amounts of less- 
toxic chemicals. The RSEI software, developed by the US EPA and based on TRI data, computes the health risks 
associated with toxic releases (RSEI Manual, US EPA, 2004).* 


Regulatory Compliance 

The Enforcement and Compliance History Online (ECHO) database focuses on facility compliance with US envi- 
ronmental regulations.? We used the following information regarding regulatory compliance: quarters in non- 
compliance, informal enforcement actions and formal enforcement actions. 


Reporting and Transparency 
Reporting and transparency are often used by socially responsible investors as an input measure to complement 
environmental performance data. For example, KLD considers environmental communication as a strength and 
defines it as ‘publishing substantive report, and having effective communication system to promote environmen- 
tal best practices’ (KLD website). SAM has a general environmental reporting section in its questionnaire explain- 
ing that industry experts review corporate environmental reports, but does not provide specific indicators to the 
public.’® 

In our analysis, we chose to use reporting and transparency indicators based on information available on the 
firm’s website and environmental/social responsibility reports. The methodology used follows the analysis of 
the content of corporate annual reports and websites developed by Brammer and Pavelin (2006)", and consists of 
the aggregation of the following seven indicators to represent the quality of companies’ corporate environmental 
disclosure (Belkaoui and Karpik; 1989; Brammer and Pavelin, 2006) (see Appendix B). 


1. Does the firm publish an environmental or sustainability report? 

2. If yes, is it according to the Global Reporting Initiative guidelines?” 

3. Has the CEO/president signed the environmental policy? 

4. Transparency and ease of obtaining information measured using the number of clicks from home page needed 
in order to read the environmental information or policy. 

5. Does the firm have specific and clear goals and improvement targets? 

6. Does the firm report actual performance numbers or just relative numbers? 

7. Are the firm’s reported numbers verified by a third party? 


In addition, we use the environmental reporting score measurement of the Pacific Social Index developed by 
the Roberts Environmental Center (REC) at Claremont McKenna College.” The REC combines qualitative and 


5RSEI considers the following information: the amount of chemical released, the location of this release, the toxicity of the chemical, its fate 
and transport through the environment, the route and extent of human exposure and the number of people affected (US EPA, 2004). 

° Information provided in ECHO relates to facilities regulated under the Clean Air Act stationary sources, Clean Water Act permitted discharg- 
ers (under the National Pollutant Discharge Elimination System) and Resource Conservation and Recovery Act hazardous waste sites (ECHO 
website). 

1° ŞAM Research Corporate Sustainability Assessment Questionnaire. 2007. Mixed. http://www.sam-group.com [7 May 2007]. 

"The criteria used by Brammer and Pavelin (2006) include the disclosure of an environmental policy, the existence of board-level responsibility 
for environmental matters, the description of environmental initiatives, reporting on environmental improvements, the setting of environmen- 
tal targets and the presence of an environmental audit or assessment. These criteria have been identified as indicators of best practices by the 
Global Reporting Initiative and the International Organization for Standardization. 

* Global Reporting Initiative website. http://www.globalreporting.org/Home [1 May 2007]. 

3 http://www.roberts.cmc.edu/PSI/whatthescoresmean.asp [1 March 2009]. 


Copyright © 2010 John Wiley & Sons, Ltd and ERP Environment Bus. Strat. Env. 19, 245-260 (2010) 
DOI: 10.1002/bse 


Measuring Corporate Environmental Performance: the Trade-Offs of Sustainability Ratings 251 


quantitative measurements to examine the quality of environmental reporting using measures similar to the ones 
we used in our analysis. These include the description of environmental issues and initiatives to address these 
issues, the existence of measurement metrics, explicit numerical goals and recognition from third parties.“ The 
score is based on the percentage of issues that were covered and how well they were covered. We find that our 
disclosure scores are highly and significantly correlated with the REC reporting scores (0.76); this correlation 
provides assurance that our methodology is consistent with other disclosure measurements.” 


Comparison of Toxic Releases, Compliance and Environmental Reporting 


The rankings based on the four different indicators (TRI, RSEI, ECHO and reporting) are presented in Table 2. 
For each indicator, companies are ranked from 1 to 15. Consistent with most screening methodologies that classify 
companies in three or more categories, we group the companies into best, middle and worst performers, and 
illustrate these using different shades of gray."° White represents the best performance (rank 1-5), light gray rep- 
resents the mid performance (rank 6-10) and dark gray represents the worst performance (rank 11-15). 

When comparing the ranking of the companies using the different indicators, we observe some drastic differ- 
ences. Five companies (Avon, Clorox, Dow, DuPont and Merck & Co.) had dissimilar rankings according to the 


Firm Rank based on Rank based ECHO (average Reporting REC KLD total KLD total 


TRI total release/ on RSEl risk non-compliance (7 criteria) ER? strengths concerns 
sales (Ib/$) score/sales quarters/facility) 
Avon Products, Inc. 1 1 6 8 1 
Clorox Company 3 7) 2 8 1 
Colgate-Palmolive 2 6 5 5 8 1 
Company 


Dial Corporation 
Dow Chemical 
Company 
DuPont Company 
Eastman Chemical 
Company 
Ecolab Inc. 


International Flavors 
& Fragrances Inc. 

Johnson & Johnson 

Lilly (Eli) and 
Company 

Merck & Co., Inc. 

Pfizer, Inc. 


Procter & Gamble 
Company 


Rohm and Haas 10 8 7 10 6 7 
Company 


Table 2. Rankings — comparison’ 
"Rank = 1 represents the best performer; rank = 15 represents the worst performer. 
? Roberts Environmental Center (REC) environmental reporting scores (ER). 


“Our seven-criterion ranking is based on additional criteria such as level of management involvement and the use of GRI guidelines. 
®The results of the different evaluations of the environmental impacts and disclosure are reported in Appendices A and C. 
16 See for example Innovest Strategic Value Advisors http://www.csrwire.com/News/8820.html [1 March 2009]. 
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metrics chosen. Overall, 14 firms moved from one category (i.e. best, mid or worst performance) to another at least 
once. For example, Clorox ranked second for compliance but 14th for reporting. Even within the indicators repre- 
senting toxic releases there were important differences. Six firms that were ranked in one category based on their 
TRI score were ranked in another category based on their RSEI score. The most striking difference, though, was 
between environmental performance and reporting indicators. Two firms that were ranked best based on the 
environmental reporting criteria were ranked worst on their toxic releases (Dow and DuPont). On the other hand, 
two firms that ranked best on TRI were ranked worst based on the reporting criteria (Avon and Clorox). 

Our results show that some firms might have high environmental output in terms of toxic releases but excellent 
environmental reporting systems. We present correlations based on these different metrics in Tables 3 and 4. Table 
3 presents correlations based on the year 2005 only while Table 4 presents the correlations for the period 2000- 
2005.7 

As presented in Tables 3 and 4, there is a positive and significant correlation between the TRI and RSEI scores 
(>0.6). The correlation is also positive and significant between the compliance and TRI and RSEI scores (>0.7). 
This indicates that firms with higher toxic releases tend to also have lower compliance levels. 

We also find that RSEI average trend data for the period 2000-2005 is not significantly correlated with RSEI 
2005 data. This indicates that one year of data might not always be representative of previous and future years 
and points to the importance of using longitudinal analysis. 

The seven-criterion reporting and CER environmental reporting scores are positively and significantly correlated 
with TRI, although they are not significantly related to the measure of TRI divided by sales. This might indicate 
that larger firms, with higher sales, are more likely to have the resources to conduct higher quality reporting. 
However, both reporting scores are positively and significantly correlated with lower levels of compliance per facil- 
ity. The CER environmental reporting scores are all significantly and positively correlated with the compliance 
scores. This indicates that firms with lower compliance are performing better on the reporting and transparency 
scores. This might be explained by the fact that companies that are flagged as being in non-compliance are the 
ones that most need to show improvement in their reporting and compliance. These results illustrate the fact that 
it is possible for companies to perform well and poorly at the same time depending on the indicators chosen. 


Comparison of Environmental Concerns Versus Environmental Strengths 


We now compare our ranking with data from KLD. KLD selects companies for the Domini 400 index that have 
positive environmental, social and governance records. KLD evaluates companies in the context of their industry 
and sector as well as in relation to the broader market. Regarding the environment, KLD includes seven strengths 
(beneficial environmental products and services; pollution prevention activities; recycling activities; use of alterna- 
tive fuels; environmental communication; above-average environmental performance for its property, plant and 
equipment and other strengths) and seven concerns (hazardous waste; regulatory problems; ozone-depleting 
chemicals; substantial emissions; agricultural chemicals; climate change and other concerns). Therefore, in the 
KLD data toxic releases are only one out of seven concerns. 

When we analyze the KLD scores for the 1991-2005 period, we find that several of the companies with higher 
environmental concern scores also have higher environmental strength scores (Dow Chemical, Dupont, Merck, 
Pfizer, Rohm and Haas). For example Dow Chemical Company has the highest number of environmental strengths 
(28) but also a very high number of environmental concerns (69) (results provided in Appendix D). 

As we can see in Table 3, there is a positive correlation between the KLD strengths and concerns for 2005 
(0.494). The correlation becomes significant at the 1 percent level for the period 2000-2005 (see Table 4). This 
is consistent with previous research showing a high correlation between environmental strengths and concerns 
within KLD (Mattingly & Berman, 2006). When looking at the correlations of KLD strengths and concerns with 
other scores, we find that both strengths and concerns are positively correlated with TRI scores. The correlations 
are significant for environmental concerns, TRI, RSEI and ECHO scores for the year 2005. In addition, in the 
2000-2005 period both environmental strengths and concerns are positively and significantly correlated with TRI 
and RSEI. However, the correlation is stronger (<o.75) for environmental concerns than for environmental 
strengths (<o.4). The KLD strengths are also positively and significantly correlated with our reporting scores 


7 The correlations are based on the actual numbers for each metric (i.e. pounds of toxic releases) and not the rankings. 
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1 2 3 4 5 6 7 8 9 10 11 12 13 
1 TRI total pounds 2005 1 
2 TRI total pounds/total sales 0.938(**) 1 
2005 


TRI total pounds/US sales 2005 0.903 (** 


3 
4 RSEI risk score 2005 0776) 
5 RSEI risk score/total sales 2005 0.657 (*) 1 
6 RSEI risk score/US sales 2005 0.644(*) 0:987) 1 
7 Ave. noncompliance Q/facility 0.837 (**) 0.780 (**) 0.481 0.437 1 
2005-2008 
8 Ave. formal actions 2003-2008/ ©0.763(**) 0.812 (**) 0.759(**) a7) 771") 1 
facility 
9 Ave. informal actions 2003- 0.780(**) 0.848 (**) 0.725(**)  0.707(**) 0.753 (**) 0.792(**) 1 
2008/facility 
10 Reporting score (score based on ©0.624(*) 0.448 0.381 0.381 0.250 0.239 0.686(**) 0.443 0.327 1 
all 7 criteria) 2007 
11 PSI env. reporting score 2006 0.714(**) 0.478 0.423 0.275 0.060 0.044 0.761 (**) 0.499 0.480 0.601(*) 1 
12 RSEI trend 2000-2005 0.411 0.363 0.385 0.495 0.433 0.530 0.165 0.090 0.278 0.228 0.231 1 
13 KLD total No concerns 2005 0.961(**) 0.881 (**) 0.831 (**) 0.799(**)  0.649(*) 0.631 (*) 0.840(**)  0.802(**)  ©0.815(**) ©0.630(*) ©0.730(**) 0.342 1 


14 KLD total No strengths 2005 0.478 0.435 0.471 0.399 0.345 0.345 0.425 0.509 0.155 0.644(*) 0.332 0.033 0.494 


Table 3. Correlations 
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1 1 2 3 4 5 6 
1 TRI_TP TRI total pounds (2000-2005) 1 
2 TRI_Size TRI total pounds/total sales (2000-2005) 1 
3 RSEl_score RSEI risk score (2000-2005) 0.728** 1 
4 RSEl_size RSEI risk score/total sales (2000-2005) 0.734** 0.957** 1 
5 KLD_Concerns total no. of concerns (2000-2005) 0.816** 0.748** 0.683** 1 
6 KLD_Strengths total no. of strengths (2000-2005) 0.289** 0.468** 0.401** 0.413%* 1 


Table 4. Correlations 
* Spearman correlations reported. Correlation is significant at the 0.05 level (two tailed). ** Correlation is significant at the 0.01 
level (two tailed). 


(0.644). In summary, firms that are better at reporting and have the most advanced environmental management 
practices are also those with higher levels of toxic releases and lower environmental compliance. Again, this illus- 
trates the trade-offs that investors face when using a limited set of indicators. If they focus exclusively on environ- 
mental management practices, they might miss firms’ current environmental impact. 

We recognize that our analysis is not without limitations. TRI provides facility information only for facilities 
located in the United States that emit above a specific threshold. It excludes small facilities. The information on 
toxic releases and compliance is therefore US based, while most of the companies studied operate internationally. 
The reporting rankings are based on the firms’ reports for 2005 and access to their websites in May 2007. This 
means that we compared data from different years for different indicators. 


Discussion 


Our findings show that the same firms can perform both well and poorly on indicators used in sustainable ratings. 
These findings mean that investors face trade-offs when choosing their screening methodology (positive or nega- 
tive screening), when choosing indicators and when assigning weights to these indicators. Based on this discussion 
and the findings of our rating case study, we provide below some principles that can help manage some of the 
trade-offs associated with sustainable rating. 


Positive or Negative Screening? 


We mentioned that the ‘best in class’ approach runs the risk of including companies that might be the worst 
performers on some dimensions. Indeed, focusing only on ‘strengths’ would mean including companies that may 
also have high concerns. This is because environmental strengths and environmental concerns are positively cor- 
related. It is therefore imperative to combine both positive and negative indicators. One simple way to conduct the 
analysis is to follow the method of Kempf and Osthoff (2007), who transformed KLD concerns into strengths by 
taking the binary complements. This way, concerns and strengths are aggregated and have the same weights. 
However, the weights to assign to positive and negative indicators need to be aligned with each rater’s main objec- 
tives regarding their preferences for environmental criteria and how these relate to financial performance. Another 
approach could be to use the Data Envelopment Analysis methodology, in which multiple indicators on a different 
scale can all be factored into one efficiency relative score (Kuosmanen and Kortelainen, 2005; Chen and Delmas, 
2009; Chen et al., 2009). 


Assigning Weights to Environmental Performance Indicators 


When using a multi-criterion methodology, the weight assignments should be clear. Although we did not illustrate 
this with our data, it is worth noting that the weight of an indicator in a rating scheme is the result of the combi- 
nation of both the weighting attributed to the indicator and of the scale that is used to measure the indicator. It 
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is therefore important that the scales measuring the indicators be similar. In addition, it is possible that not all 
the rating schemes use simple linear scoring. That is to say, criteria might not all be totally independent from each 
other. Because of this, raters should be clear on the criteria and the weighting schemes, as well as the scales used. 

The reasoning for using specific indicators and for assigning higher weights to specific indicators must be based 
on explicit priorities whenever possible. For example, grassroots organizations can argue that the actual impact 
of the companies on the local environment is more important than the reporting of their impact, and therefore 
assign higher weight to RSEI and ECHO indicators than to reporting indicators. Because the weight assignment 
is somewhat subjective, it should be carefully explained. One possible way to provide a better scientific basis for 
assigning weights is to survey stakeholders’ preferences for specific environmental impacts. For example, contin- 
gent valuation surveys could be used to assess these preferences. 

Other methodologies that have been used for life cycle assessment could also be useful (Powel et al., 1997). For 
example, emissions can be weighted based on legal limits and aggregated within each environmental medium (air, 
water, soil). Valuation can also be based on ecological scarcities. In that case the valuation is based on emission 
flows and resources relative to the ability of the environment to assimilate the flows or the extent of resources 
available. 


Choosing Indicators Wisely 


Because quantitative data on corporate environmental performance is seldom publicly available, the risk is to 
choose variables based on their availability. If investors rely only on one set of data (environmental reporting or 
toxic releases, for example), they might get an inaccurate picture of the performance of corporations. This is 
because, as we illustrated, strengths on some criteria might be correlated with weaknesses on others. This dem- 
onstrates the need for multi-criterion indicators and sometimes for detailed data that is not always publicly avail- 
able. The three categories of indicators that we described (environmental impact, regulatory compliance, and 
management systems and reporting) should be included, as they represent different facets of firm compliance. 
One example of management practice indicators can be found in the work of Delmas and Toffel (2008a, 2008b). 
The indicators used by Delmas and Toffel include, for instance, the extent of in-firm training related to corporate 
environmental performance, and the use of environmental performance in employee job-performance reviews. 


Choosing the Appropriate Comparison Group 


When comparing companies, the sample group should be carefully considered. It is important to make sure that 
the companies are comparable and that there is data available for all of them. Since companies vary in the nature 
of their operations, comparing companies from different industries might not be appropriate. In our case study 
we included companies from the chemical industry (SIC Code 28). However, the companies included came from 
different sub-sectors of the chemical industry. For example, Colgate-Palmolive is from the soap, cleaner and toilet 
goods sector (SIC 284), while Dow Chemical Company is from the plastics, materials, synthetic resins and synthetic 
rubber sector (SIC 282). In our sample, the average TRI releases in pounds per unit of sale is 1.8 lb/$ for SIC 284 
against 12.25 lb/$ for SIC 282 (see Appendix A). Differences in the amount of toxic chemicals produced might be 
a function of differences in production processes. If investors seek to keep the less polluting companies within a 
broad range of sectors, screening out the biggest polluters might be equivalent to screening out some entire sectors. 


Favoring a Longitudinal Approach 


If the ranking aims at forecasting long-term performance, it should combine indicators that are based on multi- 
year data. Improvement in environmental performance is a long process. It might take a few years before changes 
can be implemented. Looking at trends provides insights into companies’ commitment to improvement of their 
environmental performance. RSEI data is a perfect example of multi-year data that is publicly available and appro- 
priate for use when evaluating companies. As we showed, looking at static data for a specific year for all companies 
is also valuable; however, it is problematic in the case where a firm had a one-time incident that year or when data 


*® Note that not assigning weights is equivalent to assigning equal weights to the various metrics used. 
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is not available for the same year for all companies. Our analysis showed no significant correlation between a trend 
variable representing 5 years of data for RSEI and the data for the year 2005. A combination of indicators that are 
based on both single- and multi-year data is more comprehensive and robust. 

In conclusion, if rating reduces information asymmetry between what corporations know about environmental 
performance and what investors want to know, we need to better understand what goes into the rating. Transpar- 
ency about screening methodologies will facilitate methodological improvements. By comparing and evaluating 
transparent sustainability ratings, stakeholders can help improve the quality of these ratings. We propose the fol- 
lowing recommendations to increase the credibility of sustainability ratings. 


e Be clear about the goal of the evaluation. 

e Be transparent on the criteria used. 

e Use both negative and positive screening. 

e Use a methodology that reflects both real environmental impact and management practices. 
e Be specific on the comparison group. 

e Favor a longitudinal approach. 


Conclusion 


In this paper, we ranked 15 companies using three sets of indicators; the results demonstrate the challenges asso- 
ciated with screening companies based on their environmental performance. 

The main findings of our analysis of the environmental performance of 15 chemical companies is that firms 
with lower environmental performance and compliance tend to provide better quality of environmental reporting 
and to adopt more pollution prevention activities. Ratings that favor companies with better environmental manage- 
ment practices might therefore also reward companies with current lower environmental and compliance perfor- 
mance. The results of the ranking based on three main categories of indicators (impact, compliance and 
management) clearly demonstrate the drawbacks of using only one type of indicator either because of data avail- 
ability or because of investors’ preferences. The same is true when using only positive or negative indicators. 

Because there are many choices that need to be made when choosing a metric, and because these choices have 
a significant impact on the final rating, we pointed out the importance of being transparent about these choices. 
Sustainability rating methodologies are still in their early phases of development and are often contentious. Because 
there are so many ways to evaluate firms’ environmental performance and because of the current lack of standards 
in this area, there is the risk that investors might lose confidence in the approach. Transparency will increase the 
credibility of sustainability ratings and will also facilitate the standardization and potential diffusion of these 
ratings. Although we focused on corporate environmental performance, similar issues would be raised for the 
measurement of social performance. 

However, there are limits to how transparent rating organizations can be. Because there is still very limited 
publicly available information on corporate environmental performance, rating organizations invest in costly 
research about environmental management practices and performance. They must keep this information propri- 
etary in order to sell it to investors. Furthermore, each of these rating organizations develops proprietary rating 
methodology, which is the basis of their competitive advantage in the rating market. Transparency about their 
methodology would mean that they could be easily imitated by their competitors and that they could lose their 
competitive edge. Further research could investigate how sustainability rating standards could be established to 
harmonize the different existing systems, while maintaining the incentives for sustainability rating companies to 
invest in research. 

Transparency is key and well established within the financial reporting scheme. Lowenstein (1996) argues that 
the openness about the way firms operate in the United States makes the American industry more efficient and 
competitive. If this is correct for environmental performance as well, then once reporting and measuring environ- 
mental performance become more common some standardization and ‘rules’ for transparency will be needed. 
However, this will only be feasible if information about corporate environmental performance becomes publicly 
available, as is financial information. 
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Appendix A. Ranking-Based Toxic Releases, Risk Screening Environmental Indicators and 
ECHO (US EPA)’ 


Firm Rank based on TRI total Rank based RSEI risk ECHO (average 


TRI total release on RSEl risk score 2005/ non-compliance 
release 2005/ 2005/sales score 2005/ sales quarters/facility) 
sales (Ib/$) (Ib/$) sales 2005-2008 
Avon Products, Inc. 1 0.00 1 0.00 6 
Clorox Company 3 5.36 7 0.21 2 
Colgate-Palmolive Company 2 1.90 6 0.10 5 
Dial Corporation N/A N/A N/A N/A 1 
Dow Chemical Company 13 4 250.86 12 9.54 15 
DuPont Company 14 11 018.96 14 27.52 14 
Eastman Chemical Company 11 2 372.34 B 25.65 12 
Ecolab Inc. 4 55-98 5 0.081 7 
International Flavors & Fragrances Inc. 9 1 433.93 4 0.033 3 
Johnson & Johnson 5 96.95 2 0.003 9 
Lilly (Eli) and Company 7 1 366.01 8 0.508 11 
Merck & Co., Inc. 8 1 430.23 3 0.029 13 
Pfizer, Inc. 12 2 515.47 9 0.957 10 
Procter & Gamble Company 6 2 188.86 10 1.500 4 
Rohm and Haas Company 10 3 255.24 11 5.403 8 
*Rank = 1 represents the best performer; rank = 14 represents the worst performer. 
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Appendix B. Environmental Reporting Indicators?” 


Environmental Report to GRI Number of clicks Existence of Level of Reporting Third party 7-criterion 
report? standards*? to environmental performance commitment ‘hard’ verification/ score 
(Y or N) (Y or N) info. (2 or less) targets (who sign numbers auditing summary 
policy) 
Avon Products, Inc. o fe) 1 1 1 (0) 1 4.0 
Clorox Company o o 1 fe) fe) o o 1.0 
Colgate-Palmolive Company 1 o 1 fe) 1 1 1 5.0 
Dial Corporation 1 1 1 o 1 1 o 5.0 
Dow Chemical Company 1 1 1 1 1 1 oO. 6.5 
DuPont Company 1 1 1 1 1 1 1 7-0 
Eastman Chemical Company 1 o 1 o 1 1 4.0 
Ecolab Inc. 1 1 1 o 1 1 1 6.0 
International Flavors & o o o o 1 o o 1.0 
Fragrances Inc. 
Johnson & Johnson 1 1 1 1 1 1 o 6.0 
Lilly (Eli) and Company 1 1 1 1 1 1 o 6.0 
Merck & Co., Inc. 1 1 1 1 1 1 (0) 6.0 
Pfizer, Inc. 1 o 1 1 1 1 o 5.0 
Procter & Gamble Company 1 1 fe) 1 1 1 o 5.0 
Rohm and Haas Company 1 o 1 o 1 1 1 5.0 


31 = yes, O = no; data was collected for the year of 2005, but in a few cases we used data for 2003, 2004 or 2006. 
P Avon website, http://www.avon.com [1 May 2007]; Clorox website, http://www.clorox.com/ [27 April 2007]; Colgate website, http://www.colgate.com [3 May 2007]; 
DuPont website, http://www.dupont.com [1 May 2007]; Eastman Chemical website, http://www.eastman.com [1 May 2007]; Ecolab Inc. website, http://www.ecolab. 
com [2 May 2007]; Eli Lilly website, http://www.lilly.com [2 May 2007]; Henkel website, http://www.henkel.com; International Flavors & Fragrances website, http:// 
www.iff.com [2 May 2007]; Johnson & Johnson website, http://www.jnj.com [2 May 2007]; Merck website, http://www.merck.com [2 May 2007]; Pfizer website, 
http://www.pfizer.com [2 May 2007]; Procter & Gamble website, http://www.pg.com [2 May 2007]; Rohm & Haas website, http://www.rohmhaas.com [2 May 


2007]. 


*Based on GRI Registry database. 
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Appendix C. Environmental Information Disclosure Scores and Rankings: Seven-Criterion 
Score, Roberts Environmental Center (REC) Environmental Reporting Scores (ER) and Carbon 
Disclosure Project (CDP)* 


7-criterion Rank REC Rank CDP 2006 CDP 2006 Rank CDP 


score 7-criterion ER REC ER participation leadership 2006 
score scores 

Avon Products, Inc. 4.0 12 N/A N/A o -— - 
Clorox Company 1.0 14 2 12 o - 

Colgate-Palmolive Company 5.0 7 48 5 1 50 7 
Dial Corporation (The) 5.0 7 o 14 o - - 
Dow Chemical Company 6.5 2 62 3 1 85 2 
DuPont Company 7.0 1 66 1 fe) - - 
Eastman Chemical Company 4.0 12 34 9 fe) - - 
Ecolab Inc. 6.0 3 17 11 fe) - - 
International Flavors & 1.0 14 1 13 o - - 

Fragrances Inc. 

Johnson & Johnson 6.0 3 35 8 1 75 3 
Lilly (Eli) and Company 6.0 3 53 4 1 60 5 
Merck & Co., Inc. 6.0 3 42 7 1 65 4 
Pfizer, Inc. 5.0 7 44 6 1 go 1 
Procter & Gamble Company 5.0 7 65 2 1 55 6 
Rohm and Haas Company 5.0 7 27 10 o - = 


*Rank = 1 represents the best performer; rank = 14 represents the worst performer. 


Appendix D. KLD 1991-2005 


KLD: 1991-2005 
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